API reference for cpprb

cpprb Package

cpprb: Fast Flexible Replay Buffer Library

cpprb provides replay buffer classes for reinforcement learning. Details are described at Home Page.

Examples

Replay Buffer classes can be imported from cpprb package.

>>> from cpprb import ReplayBuffer

These buffer classes can be created by specifying env_dict.

>>> buffer_size = 1e+6
>>> env_dict = {"obs": {}, "act": {}, "rew": {}, "next_obs": {}, "done": {}}
>>> rb = ReplayBuffer(buffer_size, env_dict)

When adding transitions, all values must be passed as keyword arguments.

>>> rb.add(obs=1, act=1, rew=0.5, next_obs=2, done=0)

You can also add multiple transitions simultaneously.

>>> rb.add(obs=[1,2], act=[1,2], rew=[0.5,0.3], next_obs=[2,3], done=[0,1])

At the episode end, users must call on_episode_end() method.

>>> rb.on_episode_end()

Transitions can be sampled according to these buffer’s algorithms (e.g. random).

>>> sample = rb.sample(32)

Functions

create_before_add_func(env)

Create function to be used before ReplayBuffer.add

create_buffer(size[, env_dict, prioritized])

Create specified version of replay buffer

create_env_dict(env, *[, int_type, float_type])

Create env_dict from Open AI gym.space for ReplayBuffer constructor

train(ReplayBuffer buffer, env, get_action, ...)

Train RL policy (model)

Classes

HindsightReplayBuffer(size, env_dict, ...[, ...])

Replay Buffer class for Hindsight Experience Replay (HER)

LaBERlazy(batch_size[, m, eps])

Helper class for Large Batch Experience Replay (LaBER)

LaBERmax(batch_size[, m, eps])

Helper class for Large Batch Experience Replay (LaBER)

LaBERmean(batch_size[, m, eps])

Helper class for Large Batch Experience Replay (LaBER)

MPPrioritizedReplayBuffer(size[, env_dict, ...])

Multi-process support Prioritized Replay Buffer class to store transitions with priorities.

MPReplayBuffer(size[, env_dict, ...])

Multi-process support Replay Buffer class to store transitions and to sample them randomly.

PrioritizedReplayBuffer(size[, env_dict, ...])

Prioritized Replay Buffer class to store transitions with priorities.

ReplayBuffer(size[, env_dict, next_of, ...])

Replay Buffer class to store transitions and to sample them randomly.

ReverseReplayBuffer(size[, env_dict, stride])

Replay Buffer class for Reverse Experience Replay (RER)

Class Inheritance Diagram

Inheritance diagram of cpprb.HER.HindsightReplayBuffer, cpprb.LaBER.LaBERlazy, cpprb.LaBER.LaBERmax, cpprb.LaBER.LaBERmean, cpprb.PyReplayBuffer.MPPrioritizedReplayBuffer, cpprb.PyReplayBuffer.MPReplayBuffer, cpprb.PyReplayBuffer.PrioritizedReplayBuffer, cpprb.PyReplayBuffer.ReplayBuffer, cpprb.PyReplayBuffer.ReverseReplayBuffer

Indices and tables